y 



On Page 1 14, amend paragraph 4 as follows: 



Fig, 30 is Figs. 30-1 through 30*4, taken together, set forth a schematic representation of 
the camera-based object identification and attribute acquisition subsystem of Fig. 27, illustrating 
the system architecture of the slave units in relation to the master unit, and that (1) the package 
height, width, and length coordinates data and velocity data elements (computed by the LDIP 
subsystem within the master unit) are produced by the master unit and defined with respect to the 
global coordinate reference system, and (2) these package dimension data elements are 
transmitted to each slave unit on the data communication network, converted into the package 
height, width, and length coordinates, and used to generate real-time camera control signals 
which intelligently drive the camera subsystem within each slave unit, and (3) the package 
identification data elements generated by any one of the slave units are automatically transmitted 
to die master slave unit for time-stamping, queuing, and processing to ensure accurate package 
dimension and identification data element linking operations in accordance with the principles of 
the present invention; 

v 

On Page 115, amend paragraph 8 as follows: 



Fig. 33C is Figs. 33C1 and 33C2. taken together, set forth a system block diagram 
illustrating the system architecture of the bioptical PLIIM-based product dimensioning, analysis 
and identification system of the first illustrative embodiment shown in Figs. 33A and 33B; 



On Page 137, amend paragraph 2 as follows: 



Fig. 6 8 io Figs. 68-1 through 68-3. taken together, set forth a schematic block system 
diagram of a first illustrative embodiment of the airport security system of the present invention 
shown comprising (i) a passenger screening station or subsystem including PLIIM-based 
passenger facial and body profiling identification subsystem, hand-held PLIIM-based imagers, 
and a data element linking and tracking computer, (ii) a baggage screening subsystem including 
PLIIM-based object identification and attribute acquisition subsystem, a x-ray scanning 
subsystem, and a neutron-beam explosive detection subsystems (EDS), (iii) a Passenger and 
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Baggage Attribute Relational Database Management Subsystems (RDBMS) for storing co- 
indexed passenger identity and baggage attribute data elements (i.e. information files), and (iv) 
automated data processing subsystems for operating on co-indexed passenger and baggage data 
elements (i.e. information files) stored therein, for the purpose of detecting breaches of security 
during and after passengers and baggage are checked into an airport terminal system; 




On Page 295, amend paragraphs 2, 3 and 5 as follows: 



As will be described in greater detail hereinafter, the camera control computer 22 controls 
the auto-focus/auto-zoom digital camera subsystem/3" in an intelligent manner using the real- 
time camera control process illustrated in Figs. /l 8 A and 1 8 B 18A, 18B-1 and 18B2 . A 
particularly important inventive feature of this cdnera process is that it only needs to operate on 
one data set at time a time, obtained^gmJb^DIP Subsystem 122, in order to perform its 
complex array of functions. Referring ro Figs. i % A and 1 8 B 18 A. 18B-1 and 18B2. the real-time 
camera control process of the illustrative embodiment will now be described with reference to 
the data structures illustrated in Figs, 18jjgdi&fffi&the data tables illustrated in Figs. 21 and 23. 

Real-Time Camera Control Process OftTM Present Invention 

In the illustrative embodiment, the Real-time Camera Control Process 560 illustrated in 
Figs. 1 8 A and 18B 18 A. 18B-1 and 18B2 is carried out within the camera control computer 21 of 
the PUIM-based system 120 shown in Fig. 9. It is understood, however, that this control process 
can be carried out within any of the PLIIM-based systems disclosed herein, wherein there is a 
need to perform automated realtime object detection, dimensioning and identification 
operations. II 

As illustrated in Figs. 1 8 jA - and - 1 8 B 18 A. 18B-1 and 18B2. the camera control process of 
the present invention has multiple control threads that are carried out simultaneously during each 
data processing cycle (i.e. erch time a new data set is received from the Real-Time Package 
Height Profiling And Edge Detection Processing Module 550 within the LDIP subsystem 122). 
As illustrated in this flowchart, the data elements contained in each received data set are 
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automatically processed within the camera control computer in the manner described in the flow 
chart, and at the end of each data set processing cyclef generates real-time camera control signals 
that drive the zoom and focus lens group transtgta&powered by high-speed motors and quick- 
response linkage provided within high-speed auto-focus/auto-zoom digital camera subsystem 
(i.e. the IFD module) 3" so that the camera s*tesystgm 3" automatically captures digital images 
having (1) square pixels (i.e. 1:1 aspect n|Ko>f independent of package height or velocity, (2) 
significantly reduced speckle-noise levels, ami (3) constant image resolution measured in dots 
per inch (DPI) independent of package height or velocity. Details of this control process will be 
described below. / 





On Page 297, ai^fendparagraph 7 as follows: 




> 


At Block D in Fig. 18 A, at this stage in the/control process, the camera control computer 
22 analyzes the height values (Le. coordinates) buffered in the Package Data Buffer, and 
determines the current "median" height of the package. At this stage of the control process, 
numerous control "threads'* are started, each parrying out a different set of control operations in 
the process. As indicated in the flow chart of Fies. 18A and 18B 18 A. 18B-1 and 18B2. each 
control thread can only continue wnfen iHe'necessaiy parameters involved in its operation have 
been determined (e.g. computed), and thusfthe control process along a given control thread must 
wait until all involved parametewprf^^ before resuming its ultimate operation (e.g. 
computation of a particular intermediate parameter, or generation of a particular control 
command), before ultimately retumingfto the start Block A, at which point the next time-stamped 
data set is received from the Reafl-Time Package Height Profiling And Edge Detection 
Processing Module 550. In the illustrative embodiment, such data set input operations are 
carried out every 5 milliseconds, And therefore updated camera commands are generated and 
provided to the auto-focus/autojfzoom camera subsystem at substantially the same rate, to 
achieve real-time adaptive camera control performance required by demanding imaging 
applications. / 






/ 

On Page 298, amend paragraphs 1 and 6 as follows: 
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As indicated at Blocks E, F, G H, I, A in Figs. 18A and 18B 18A. 18B-1 and 18B2. a first 
control thread runs from Block D to Block A so as to reposition the focus and zoom lens groups 
within the auto-focus/auto-zoom digital camera subsvfctem each time a new data set is received 
from the Real-Time Package Height Profiling And E<fge Detection Processing Module 550. 



As indicated at Blocks D, K, L, M in 
control thread runs from Block D in order t 
time period (ATphoto-mtegratjon) parameter whi 
auto-focus/auto-zoom digital camera subsys$mj 



f 8 A andl 8 B 18A. 18B-1 and 18B2. a second 
^ermine and set the optimal photo-integration 
; that digital images captured by the 
ill have pixels of a square geometry (i.e. aspect 



ratio of 1:1) required by typical image-based bar code symbol decode processors and OCR 
processors. As indicated at Block K, the camera control computer analyzes the current median 
height value in the Data Package Buffer, md determines the speed of the package (Vb). At 
Block L, the camera control computer uses |ne computed values of average (i.e. median) package 
height, belt speed and Photo-Integration#rime Look-Up Table in Fig. 22B, to determine the 
photo-integration time parameter (ATphDtJLegration) which will ensure that digital images captured 
by the auto-focus/auto-zoom digital camera subsystem will have pixels of a "square" geometry 
(i.e. aspect ratio of 1 : 1). 



Reference will now be made to Fig. 18D and 18E1 and E2 in order to explain the camera 
line rate compensation operation of the present invention carried out at Block L in Fig. 1 8 B Figs. 
18B-1 and 18B-2 . Notably, the primary purpose of this operation is to automatically compensate 
for viewing-angle distortion which would otherwise occur in images of object surfaces captured 
as the object surfaces move past the coplanar PLIB/FOV of PLIIM-based linear 25* at skewed 
viewing angles, defined by slope angles 6 and <}> in Figs. 18E1 and 18E2, for the cases of top 
scanning and side scanning, respectively. 



On Page 300, amend paragraphs 1 and 2 as follows: 



Page 5 of 12 




In a PLIIM-based linear imaging system, configured above a conveyor belt structure as 
shown in Fig. 18E1, the Line Rate of the linear image detection array in the camera subsystem 
will be dynamically adjusted in accordance with the principles of the present invention described 
above. In this case, the method employed at Block L in Fig. - 1 8 B Figs. 18B-1 and 18B-2 and 
detailed in Fig. 18D will provide a high level of compensation for viewing angle distortion 
presented when imaging (the plane of) a moving object surface disposed skewed at some slope 
angle B measured relative to the planar surface of the conveyor belt. In this case, the difficulty 
will should not reside in line-rate compensation, but rather in dynamically focusing the image 
formation optics of the camera (IFD) subsystem in response to the geometrical characteristics of 
the top surfaces of packages measured by the LDIP subsystem (i.e. instrument) 122 on a real- 
time basis. For example, during illumination and imaging operations, a slanted or sloped top 
surface of a transported box or object must remain in focus under the camera subsystem. To 
achieve such focusing, the slope of the object's top surface should be within a certain value, 
across the entire conveyor belt However, in the top scanning case, if the box is rotated along the 
direction of travel so that the slope of the top surface thereof is not substantially the same across 
the conveyor belt (i.e. the height values of the box vary across the width of the conveyor belt), 
then it will be difficult for the camera subsystem to focus on the entire top surface of the box, 
across the width of the conveyor belt. In such instances, the LDIP subsystem 122 in system 120 
has the option (at Block L in Fig. 18B Figs. 18B-1 and 18B-2) of providing only a single height 
value to the camera control computer 22 (e.g. the average value of the height values of the box 
measured across the conveyor belt), and for this average value to be used by the camera control 
computer 22 to adjustably control the camera's zoom and focus characteristics. Alternatively, the 
LDIP subsystem 122 can transmit to the camera control computer 22 , data representative of the 
actual slope and shape of the top surface of the box, and such data can be used to control the 
focusing optics of the camera subsystem in a more complicated manner permitted by the image 
forming optics used in the linear PLIIM-based imaging system. 

For the case of side scanning shown in Fig. 18E2, the method of the present invention 
employed at Block L in Fig. 1 8 B Figs. 18B-1 and 18B-2 and detailed in Fig. 18D will provide a 
high level of compensation for viewing angle distortion which will otherwise occur in images of 
object surfaces when viewing (the plane of) the moving object surface disposed skewed at some 
angle $ measured relative to the edge of the conveyor belt. 
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On Page 301, amend paragraphs 1, 2, 4 and 5 as follows: 



Referring back now to Block M in Fig. 1 8 B Figs. 18B-1 and 18B-2. it is noted that the 
camera control computer 22 generates a digital control signals for the parameters (1) Photo- 
integration Time Period (ATphoto-integretion) found in the Photo-Integration Time Look-Up Table set 
forth in Fig. 1 822B, and (2) the Compensated Line Rate parameter computed using the procedure 
set forth in Fig. 18D. Thereafter, the camera control computer 22 transmits these digital control 
signals to the CCD image detection array employed in the auto-focus/auto-zoom digital camera 
subsystem (i.e. the IFD Module). Thereafter, this control thread returns to Block A as indicated 
in Fig. 18A. 

As indicated at Blocks D, N, O, P, R in Figs. 18A and 1 8 B 18A. 18B-1 and 18B-2 . a third 
control thread runs from Block D in order to determine the pixel indices (i j) of a selected portion 
of a captured image which defines the "region of interest" (ROI) on a package bearing package 
identifying information (e.g. bar code label, textual information, graphics, etc.), and to use these 
pixel indices (ij) to produce image cropping control commands which are sent to the image 
processing computer 21. In turn, these control commands are used by the image processing 
computer 21 to crop pixels in the ROI of captured images, transferred to image processing 
computer 21 for image-based bar code symbol decoding and/or OCR-based image processing. 
This ROI cropping function serves to selectively identify for image processing only those image 
pixels within the Camera Pixel Buffer of Fig. 20 having pixel indices (i J) which spatially 
correspond to the (row, column) indices in the Package Data Buffer of Fig. 19. 

At Block O in Fig. 1 8 B Figs. 18B-1 and 18B-2 . the camera control computer detects the 
x coordinates of the package boundaries based on the spatially transformed coordinate values of 
the left and right package edges (LPE,RPE) buffered in the Package Data Buffer, shown in Fig. 
19. 

At Block P in Fig. 1 8 B Figs. 18B-1 and 18B-2, the camera control computer 22 
determines the corresponding pixel indices (ij) which specifies the portion of the image frame 
(i.e. a slice of the region of interest), to be effectively cropped from the image to be subsequently 
captured by the auto-focus/auto-zoom digital camera subsystem 3". This pixel indices 
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specification operation involves using (i) the x coordinates of the detected package boundaries 
determined at Block O, and (ii) optionally, the subrange of x coordinates bounded within said 
detected package boundaries, over which maximum range "intensity" data variations have been 
detected by the module of Fig. IS. By using the x coordinate boundary information specified in 
item (i) above, the camera control computer 22 can determine which image pixels represent the 
overall detected package, whereas when using the x coordinate subrange information specified 
in item (ii) above, the camera control computer 22 can further determine which image pixels 
represent a bar code symbol label, hand-writing, typing, or other graphical indicia recorded on 
the surface of the detected package. Such additional information enables the camera control 
computer 22 to selectively crop only pixels representative of such information content, and 
inform the image processing computer 21 thereof, on a real-time scanline-by-scanline basis, 
thereby reducing the computational load on image processing computer 21 by use of such 
intelligent control operations. 



On Page 302, amend paragraphs 1 and 2 as follows: 



Thereafter, this control thread dwells at Block R in Fig. 1 8 B Figs. 18B-1 and 18B-2 until 
the other control threads terminating at Block Q have been executed, providing the necessary 
information to complete the operation specified at Block Q, and then proceed to Block R, as 
shown in Fig, 1 8 B Figs. 18B-1 and 18B-2 . 

As indicated at Block Q in Fig. 1 8 B Figs. 18B-1 and 18B-2. the camera control computer 
uses the package time stamp (nT) contained in the data set being currently processed by the 
camera control computer, as well as the package velocity (V b ) determined at Block K, to 
determine the "Start Time** of Image Frame Capture (STIC). The reference time is established 
by the package time stamp (nT). The Start Time when the image frame capture should begin is 
measured from the reference time, and is determined by (1) predetermining the distance Az 
measured between (i) the local coordinate reference frame embedded in the LDIP subsystem and 
(ii) the local coordinate reference frame embedded within the auto-focus/auto-zoom camera 
subsystem, and dividing this predetermined (constant) distance measure by the package velocity 
(Vb). Then at Block R, the camera control computer 22 (i) uses the Start Time of Image Frame 
Capture determined at Block Q to generate a command for starting image frame capture, and (ii) 
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uses the pixel indices (i j) determined at Block P to generate commands for cropping the 
corresponding slice (i.e. section) of the region of interest in the image to be or being captured 
and buffered in the Image Buffer within the IFD Subsystem (i.e. auto-focus/auto-zoom digital 
camera subsystem). 




As shown in Fig. 25, unitary system 140 comprises a PLIIM-based camera subsystem 
25"' which includes a high-resolution 2D CCD camera subsystem 25" similar in many ways to 
the subsystem shown in Figs. 6D1 through 6E3, except that the 2-D CCD camera's 3-D field of 
view is automatically steered over a large scanning field, as shown in Fig. 6E4, in response to 
FOV steering control signals automatically generated by the camera control computer 22 as a 
low-resolution CCD area-type camera (640x640 pixels) 61 determines the x,y position 
coordinates of bar code labels on scanned packages. As shown in Figs. 5B3, 5C3, 6B3, and 6C3, 
the components (61 A, 6 IB and 62) associated with low-resolution CCD area-type camera 61 are 
easily integrated within the system architecture of PLIIM-based camera subsystems. In the 
illustrative embodiment, low-resolution camera 61 is controlled by a camera control process 
carried out within the camera control computer 22, by modifying the camera control process 
illustrated in Figs. 18 A and 1 8 B 18 A. 18B-1 and 18B-2 . The major difference with this modified 
camera control process is that it will include subprocesses that generate FOV steering control 
signals, in addition to zoom and focus control signals, discussed in great detail hereinabove. 

^ 

On Page 315, amend paragraph 3 as follows: 



In this illustrative tunnel-type system, only the top PID unit 120 includes an LDIP 
subsystem 122 for object detection, tracking, velocity-detection and dimensioning/profiling 
functions, as this PID unit functions as a master PID unit within the tunnel system 570, whereas 
the side and bottom PID units 120 are not provided with a LDIP subsystem 122 and function as 
slave PID units. As such, the side and bottom PID units 120* are programmed to receive object 
dimension data (e.g. height, length and width coordinates) from the master PID unit 120 on a 
real-time basis, and automatically convert (i.e. transform) these object dimension coordinates 
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into their local coordinate reference frames in order to use the same to dynamically control the 
zoom and focus parameters of the camera subsystems employed in the tunnel system. This 
centralized method of object dimensioning offers numerous advantages over prior art systems 
and will be described in greater detail with reference to Figs. 3© 30-1 through 32B. 



On Page 316, amend paragraphs 1-3 as follows: 



In Figs. 30 30-1 through 32B, CCD camera-based tunnel system 570 of Fig. 27 is 
schematically illustrated employing a real-time method of automatic camera zoom and focus 
control in accordance with the principles of the present invention. As will be described in 
greater detail below, this real-time method is driven by object coordinate data and involves (i) 
dimensioning packages in a global coordinate reference system, (ii) producing object (e.g. 
package) coordinate data referenced to said global coordinate reference system, and (iii) 
distributing said object coordinate data to local coordinate references frames in the system for 
conversion of said object coordinate data to local coordinate reference frames and subsequent 
use automatic camera zoom and focus control operations upon said packages. This method of 
the present invention will now be described in greater detail below using the four-sided tunnel- 
based system 570 of Fig. 27, described above. 

As shown in Fig. 30 Figs. 30-1 through 30-4. the four-sided tunnel-type camera-based 
object identification and attribute acquisition system of Fig. 27 comprises: a single master PID 
unit 120 embodying a LDIP subsystem 122, mounted above the conveyor belt structure 571; 
three slave PID units 120*, 120* and 120*, mounted on the sides and bottom of the conveyor belt; 
and a high-speed data communications network 572 supporting a network protocol such as, for 
example, Ethernet protocol, and enabling high-speed packet-type data communications among 
the four PID units within the system. As shown, each PID unit is connected to the network 
communication medium of the network through its network controller 132 (133) in a manner 
well known in the computer networking arts. 

As schematically illustrated in Figs. 30 and 30-1 through 31, local coordinate reference 
systems are symbolically embodied within each of the PID units deployed in the tunnel-type 
system of Fig. 27, namely: local coordinate reference system Ruxaio symbolically embodied 
within the master PID unit 120; local coordinate reference system Rioc&u symbolically embodied 
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within the first side PID unit 120'; local coordinate reference system Rjocai2 symbolically 
embodied within the second side PID unit 120'; and local coordinate reference system Riocau 
symbolically embodied within the bottom PID unit 120'. In turn, each of these local coordinate 
reference systems is "referenced" with respect to a global coordinate reference system Rgbbai 
symbolically embodied within the conveyor belt structure. Object coordinate information 
specified (by vectors) in the global coordinate reference system can be readily converted to 
object coordinate information specified in any local coordinate reference system by way of a 
homogeneous transformation (HG) constructed for the global and the particular local coordinate 
reference system. Each homogeneous transformation can be constructed by specifying the point 
of origin and orientation of the x,y,z axes of the local coordinate reference system with respect to 
the point of origin and orientation of the x,y,z axes of the global coordinate reference system. 

Qits*l* sIa+oiIo stfi tt/vn^stfrwt^fYiic troticfrtimohrttic orp u/a11 IftlAwn in tJiA art 
OUCH QClallS OU nOIIlUKCUCUUi> U oil^iUl lllallUllo (UC WCll MIUWU 111 UlC all. 


— — — 


— — 


On Pace 318. amknd narasraoh 2 as follows: 






In addition. Fig. 30 Figs. 30-1 through 30-4 illustrates that the LDIP subsystem 122 
within the master unit 120 generates (i) package height, width, and length coordinate data and 
(ii) velocity data, referenced with respect to the global coordinate reference system Rgiobai. These 
package dimension data elements are transmitted to each slave PID unit 120' on the data 
communication network, and once received, its camera control computer 22 converts there 
values into package height, width, and length coordinates referenced to its local coordinate 
reference system using its preprogrammable homogeneous transformation. The camera control 
computer 22 in each slave PID unit 120 uses the converted object dimension coordinates to 
generate real-time camera control signals which automatically drive its camera's automatic zoom 
and focus imaging optics in an intelligent, real-time manner in accordance with the principles of 
the present invention. The "object identification" data elements generated by the slave PID unit 
are automatically transmitted to the master PID unit 120 for time-stamping, queuing, and 
processing to ensure accurate object identity and object attribute (e.g. dimension/profile) data 
element linking operations in accordance with the principles of the present invention. 
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